Image scanner and method for detecting a defect in an image to be scanned

ABSTRACT

Image data in an image scanner is examined to determine whether lines (light or dark) are present in the image data. By examining whether the lines are present in image data for multiple colors, and whether calibration gains for corresponding photosensors are normal, it can be determined whether the lines &amp; likely caused by a surface defect, on a calibration target, or on a platen, or on an image being scanned.

RELATED APPLICATIONS

This application is a divisional application of U.S. patent applicationSer. No. 09/845,852, filed Apr. 30, 2001, now U.S. Pat. No. 7,183,532DETECTING A DEFECT IN AN IMAGE IN AN IMAGE SCANNER.

FIELD OF INVENTION

This invention relates generally to devices for digital electronicscanning of images and more specifically to detection of dust andscratches and other surface defects.

BACKGROUND OF THE INVENTION

Electronic image scanners convert an optical image into an electronicform suitable for storage, transmission or printing. In a typical imagescanner, light from an image is focused onto line-arrays of photosensorsfor scanning one line at a time. A two dimensional image is scanned byproviding relative movement between the photosensor line-arrays and theoriginal image. In general, a color scanner measures the intensity of atleast three relatively narrow bands of wavelengths of visible light, forexample, bands of red, green and blue.

For image scanners, the digitized image may be degraded by the presenceof artifacts on the surface of the object being scanned, such as dustand fingerprints, or defects in the surface of the object being scanned,such as scratches, folds, or textured surfaces. Multiple methods havebeen disclosed for detecting defects on transparent media. See, forexample, U.S. Pat. Nos. 5,266,805, 5,969,372, and EP 0 950 316 A1. Someof the methods in the referenced patent documents utilize the fact thatthe dyes in transparent color film are essentially transparent toinfrared light, whereas dust and scratches are relatively opaque. Otherdisclosed methods utilize dark field imaging, in which the lightreaching the photosensors is reflected or diffracted by defects insteadof the film.

Scanners for opaque media are configured differently than scanners fortransmissive media, and different detection methods are needed. Commonlyassigned U.S. patent application Ser. No. 09/629,495, filed Jul. 13,2000 discloses defect detection, in an opaque media scanner, havingmultiple spaced-apart line-arrays of photosensors, where surface defectscast shadows, and the length of the shadows, as seen by each line-arrayof photosensors, varies among the different line-arrays of photosensors.

Reflective document scanners and copiers commonly provide afixed-position calibration target, along a scanline dimension. In a flatbed scanner with a motionless document being scanned, the calibrationtarget is typically beneath a glass platen in a relatively dust freeenvironment. The calibration target is used to ‘compensate, beforescanning, for variation in sensitivity of individual photosensors, andfor variation in light intensity along the length of the scanline. Theprocess is called Photo-Response Non-Uniformity (PRNU) calibration. See,for example, U.S. Pat. No. 5,285,293. Since the calibration target ispresumably uniform, any pixel to pixel intensity variation can beattributed to sensor sensitivity, light source variation, or othersystem uniformity. A correction factor (gain and/or offset) iscalculated and applied to subsequent scans. Just in case there is asurface defect on the calibration target, it is known in commerciallyavailable scanners to accumulate data from many scanlines (thephotosensors are moved relative to the calibration target) during PRNUcalibration, and to average the data on a photosensor-by-photosensorbasis. It is also known to discard extreme data points before averaging.For example, given ten intensity measurements for one photosensor, it isknown to discard the lowest and highest intensity values, and thenaverage the remaining eight values. This helps eliminate the effects ofsurface defects on the calibration target during PRNU calibration.

Of particular concern in the present application is scanners in which adocument moves past a stationary photosensor array, for example,scroll-feed scanners, and flat-bed scanners with automatic documentfeeders. Scroll-feed scanners and scanners with automatic documentfeeders have several unique problems regarding detection and correctionof surface defects. In a scroll-feed scanner, the document may bedisplaced from a platen, so the calibration target is typically behindthe document being scanned to properly measure the light at thedocument. This in turn causes three potential problems. First, thecalibration target is much more susceptible to debris introduced bydocuments, particularly paper debris. Second, the photosensor array istypically stationary, so the technique of averaging multiple scanlinesto eliminate the effects of surface defects during PRNU calibration isnot applicable. Third, if the calibration target is behind the documentbeing scanned, and if there is debris on the calibration target, thenthe PRNU calibration compensates for the debris, but subsequent documentscanning hides the debris on the calibration target so that the PRNUgain or offset is inappropriate. The result is a streak in the digitizedimage. Finally, with moving documents, it is common for debris to becometemporarily trapped between the document and a glass platen, and thenlater the debris may be dislodged. Again, the result is a streak in thedigitized image.

There is a need for surface defect detection for the unique situationspresented by moving documents: (1) debris on the calibration target witha stationary photosensor array, and (2) temporary debris on a platenduring scanning.

SUMMARY OF THE INVENTION

Narrow streaks (light or dark), corresponding to a few photosensors,appearing primarily in one color channel, are analyzed to see if thestreaks are likely caused by a surface defect, either on the calibrationtarget or on the platen. Debris on the calibration target causes thePRNU gain for some photosensors to be abnormally high. If the debris islater hidden by the document being scanned, the result is a highintensity streak for one color channel in the digitized image.Accordingly, images are searched for a high intensity streak in onecolor channel of the digitized image corresponding to an anomalous PRNUgain. Debris on the platen introduced after PRNU calibration results ina low intensity streak in one color channel, with normal correspondingPRNU gains. Accordingly, the system may also search for a low intensitystreak in one color channel of the digitized image with a normalcorresponding PRNU gain. For scanners with multiple linearrays of thesame or nearly the same color, detection is simplified. With twolinearrays of nearly the same color, if a light or dark streak appearsin data from only one of the light arrays, it is likely caused by adefect. Examining the corresponding PRNU gains may aid in determiningthe cause of the defect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram side view of a scanner with a moving document.

FIG. 2 is a flow chart of an example method for determining that adefect exists on a PRNU calibration target.

FIG. 3 is a flow chart of an example method for determining that adefect exists on a platen or on an image being scanned.

FIG. 4 is a block diagram side view of a scanner with a moving documentwith an alternative configuration for the photosensor assembly.

FIG. 5 is a flow chart of an example method for determining that adefect exists when using a photosensor assembly as in FIG. 4.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT OF THE INVENTION

In FIG. 1, a document 100 is being moved adjacent to a transparentplaten 102. The direction of movement is indicated by an arrow. Thedocument is illuminated by a lamp 104. An objective lens 106 focuseslight, scattered from scanlines on the surface of the document 100, ontoline-arrays of photosensors. A photosensor assembly 108 has threeline-arrays of photosensors (110, 112, 114), designated R, G, and B forred, green, and blue. FIG. 1 illustrates a scanner with an objectivelens, but the invention is equally applicable to contact imagingphotosensor arrays, fiber-optic imaging, and photosensor arrays usingrod lens arrays.

Light scattered by a first scanline on the document 100 is focused, bythe lens 106, onto the red line-array 110. Light scattered by a secondscan-line is focused onto the blue line-array 114. A dust particle 116(exaggerated to facilitate illustration) lies on the scanline for thered line-array 110.

A calibration target 118 is located behind the document 100 (as seen bythe photosensor assembly 108). Before the document 100 is inserted, thelamp 104 illuminates the calibration target, and PRNU calibration isperformed. Dust particle 116 may alternatively be located on thecalibration target 118 along the lime that is focused onto the redline-array 110.

Signals from the line-arrays of photosensors (110, 112, 114) aretypically amplified by at least one amplifier (not illustrated),converted to digital values by at least one analog-to-digital converter(not illustrated), and the resulting digital values are stored in amemory (not illustrated) where they may be analyzed by a processor (notillustrated). PRNU calibration determines a gain used by the system forthe signal from each photosensor, where that gain may be an analogamplifier gain, or the gain may be a digital gain that is part of acomputation on the digital values. That is, the signal from eachphotosensor has an associated unique gain determined by PRNUcalibration. The signal processing and data processing for onelime-array is commonly referred to as a color channel.

First, consider the situation in which the dust particle 116 is on thesurface of the calibration target 118, and where document 100 is notpresent. At least one photosensor in line-array 110 receives a lowintensity, and the PRNU calibration increases the gain used for theaffected photosensors. Then, when the document 100 is scanned, the PRNUgain for at least one photosensor in line-array 110 is excessively high,resulting in intensity measurements that indicate an intensity muchhigher than the actual intensity. The result is a high intensity line orstreak in the red image data (before any color transformation iscomputed), along the direction of movement of the document. Assumingthat the dust particle 116 does not affect scanlines for otherline-arrays, there is no corresponding streak in the blue or green imagedata.

The PRNU calibration data is searched to see if the PRNU gains for anyindividual photosensors exceed a predetermined threshold. For theexample of FIG. 1, the red image data, before any color transformationis computed, corresponding to the photosensors with an abnormally highPRNU gain, are examined to determine whether there is an intensity,along the dimension of the image corresponding to movement of thedocument 100, that exceeds a I predetermined threshold. If there is aphotosensor with an abnormally high PRNU gain, and a corresponding highintensity line or streak in the image data, then it is likely that therewas a surface defect on the calibration target during PRNU calibration.Note that a high intensity streak in the image data could be alegitimate part of the image. For example, a red line on the documentbeing scanned would appear as a high intensity strip in the reddigitized image data. However, by starting with high PRNU gain, andlooking only at the data corresponding to high PRNU gain, theprobability that the high intensity data represents a legitimate part ofthe image is greatly reduced.

There are multiple alternatives for compensation for the surface defect.The image data can be recomputed with a lower PRNU gain for photosensorsthat are identified as having an abnormally high PRNU gain. However, ifsome digital data values are clipped at a maximum value, then someinformation has been irretrievably lost. Alternatively, the PRNU gaincan be corrected and the image can be rescanned. Alternatively, thedigitized image may be edited to replace the pixels in the highintensity streak with a combination of the intensity values from nearbypixels. For example, a pixel can be replaced with the average value ofadjacent pixels, or bilinear interpolation using four neighboring pixelscan be used, or more general resampling techniques can be used. If theimage data is modified, the operator can be alerted that the image isbeing modified, and the operator can accept or decline the modification.Preferably, the operator is presented with an image 15 with thecorrection and an image without the correction, and the operator ispermitted to choose between the two. The system can also request thatthe operator clean the calibration target. Finally, PRNU calibration canbe performed again, and if the abnormally high gains have not changed,then the PRNU gain for the photosensors corresponding to the identifieddefect can be adjusted for subsequent scans.

Now consider the situation in which the dust particle 116 is introducedonto the platen 102 after PRNU calibration. The dust particle may bepresent for an entire scan of a document, or may appear and disappearduring the scan of a document. The result is a line or streak of lowintensity in the data for a single color channel (before any colortransformation computation). For example, in FIG. 1, there would be aline of low intensity in the red data, but not in the blue or greendata. The PRNU data for the red photosensors is examined to see if thePRNU gain, for the photosensors corresponding to the dark line, exceedsa predetermined threshold. If so, then it is likely that the dustparticle was present on the platen during PRNU calibration. If the dustparticle was present on the platen during PRNU calibration, and is stillpresent, then the dust particle is blocking the light, for at least onephotosensor, from the image being scanned, and PRNU gain adjustment isunlikely to help. In that case, the image may be edited to replace thepixels in the dark streak with a combination of the intensity valuesfrom nearby pixels. For example, a pixel can be replaced with theaverage value of adjacent pixels, or bilinear interpolation using fourneighboring pixels can be used, or more general resampling techniquescan be used. Again, the operator can be alerted and the operator candecide whether to proceed with image modification.

Note that there are other potential causes of a low intensity streak. Ablack or dark line in the image being scanned would result in a lowintensity streak in all color channels. A high intensity streak, in theimage being scanned, of one color, may result in a low intensity streakin other colors. For example, if the filters were ideal, then a greenline in the image being scanned would result in low intensity data inthe red and blue color channels. However, real filters or colorseparators typically used for image scanning have some spectral overlap,so that a green line in the image being scanned might result in arelatively low intensity for the corresponding red and bluephotosensors, but not as low as the intensity resulting from a blackline or from an occluded photosensor. Again, thresholding can be used.If a dark streak in the red data has a corresponding relatively darkstreak in only one of the green data and blue data, then the streak maybe a legitimate part of the image.

If a dark streak appears ody in one color channel, and the PRNU gainsfor the corresponding photosensors do not exceed the predeterminedthreshold, then it is likely that a dust particle appeared after thePRNU calibration Again, gain adjustment is unlikely to help, and theimage may be edited to replace the pixels in the dark streak with acombination of the intensity values from nearby pixels. FIG. 2illustrates an example method for determining that a defect appears onthe calibration target. At step 200, the PRNU gain data for one colorchannel is examined. At decision 202, if the PRNU gain for anyphotosensors exceeds a predetermined threshold, then at step 204 thecorresponding pixels in the image data (before color transformation) areexamined. At decision 206, if the intensity data for the pixelscorresponding to the high PRNU gain values also exceeds a predeterminedthreshold, then at step 208 the system determines that there is a defecton the PRNU calibration target. The process illustrated in FIG. 2 isrepeated for each abnormally high PRNU gain, as necessary, and isrepeated for all colors.

FIG. 3 illustrates an example method for determining that there is adefect on the platen that was present during PRNU calibration, or thatthere is a defect on the platen or on the image being scanned that wasnot present during PRNU calibration. Preliminary to step 300, the imagedata for one color channel (before color transformation) is examined tosee if there is a stripe, in the direction of document movement, havinga low intensity. That is, for multiple scanlines, the intensity of theNth pixel in the scanlines is less than a predetermined threshold. Atdecision 300, if a low intensity stripe has been detected, then at step304 the corresponding pixels in the image data for other color channelsare examined. At decision 306, if there is a low intensity stripe forall colors, then at step 308 the system determines that there is a darkline in the image being scanned. At decision 310, if there is a lowintensity stripe for more than one color, but not all colors, then atstep 312 the system determines that there is a color lie in the imagebeing scanned. At decision 314, it has been determined that there is alow intensity stripe in the image data for only one color. If the PRNUgain data, for photosensors corresponding to the low intensity stripe,indicate an abnormally high PRNU gain, then at step 316 the systemdetermines that there was a defect (for example, on the platen) presentduring PRNU calibration and the defect is still present. If the PRNUgain data, for photosensors corresponding to the low intensity stripe,indicate normal PRNU gains, then at step 318 the system determines thatthere is a defect on the platen or on the image being scanned. Foreither step 316 or step 318, the only effective compensation for thedefect is image editing (gain adjustment will not help). For thatreason, decision 314 is optional. Note also that the conclusion at step316 cannot distinguish between a defect on the platen and a badphotosensor, but from a practical standpoint this may not matter. Thatis, in either case, image correction may be required.

A photosensor assembly may have multiple spaced-apart line-arrays thatreceive light having the same spectral bandwidth, or almost the samespectral bandwidth. For example, a photosensor assembly may have one setof line-arrays with relatively small photosensors having a relativelyhigh sampling rate but relatively low signal-to-noise, and a separateset of line-arrays with relatively large photosensors having arelatively low sampling rate but relatively high signal-to-noise. Havingtwo different sizes of photosensors provides substantial versatility inscanning. Given a photosensor assembly with multiple rows of the samecolor, defect detection is substantially simplified. With twoline-arrays of nearly the same color, if a light or dark streak appearsin data from only one of the line-arrays, it is likely caused by adefect.

FIG. 4 illustrates an example scanner of the general type illustrated inFIG. 1, but with a photosensor assembly having multiple line-arrays ofeach color. In FIG. 4, photosensor assembly 400 has three line-arrays ofrelatively small photosensor (402,204,406), and three line-arrays ofrelatively large photosensors (408,410,412). As illustrated in FIG. 4,line-arrays 402 and 408 receive red light, line-arrays 404 and 410receive green light, and line-arrays 406 and 412 receive blue light. Twoline-arrays of the same “color” (for example, lime arrays 402 and 408)may receive identical spectral bandwidths of light, or they may receivespectral bandwidths that substantially overlap but are slightlydifferent. For purposes of the example embodiment of the invention, itdoes not matter whether the two sets of photosensors are of the samesize, or different sizes, and it does not matter whether the spectralbandwidths are identical, or almost the same.

FIG. 5 illustrates an example flow chart for a mew of detecting defectswhen the scanner has two line-arrays of nearly the same color. Indecisions 500, 502, and 504, if a streak (light or dark) appears in thedigitized intensity data for only one of two line-arrays receiving lighthaving the same or almost the same spectral bandwidths, then at step 506the streak indicates a defect. The rest of the steps illustrated in FIG.5 are optional, and may be executed if additional diagnostic informationis desired. At decision 508, if the streak is a light (high intensity)streak, then at step 510 there is likely a defect on the calibrationtarget. At decision 512, if the streak is dark (low intensity) and ifthe corresponding PRNU gains are greater than a predetermined threshold,then at step 514 there is likely a defect on the platen that was presentduring PRNU calibration. At decision 512, if the streak is dark (lowintensity) and if the corresponding PRNU gains are not greater than apredetermined threshold, then at step 516 there is likely a defect onthe platen that was not present during PRNU calibration. Note that theconclusion at step 514 cannot distinguish between a defect on the platenand a bad photosensor, but from a practical standpoint this may notmatter. That is, in either case, image correction may be required.

The foregoing description of the present invention has been presentedfor purposes of illustration and description. It is not intended to beexhaustive or to limit the invention to the precise form disclosed, andother modifications and variations may be possible in light of the aboveteachings. The embodiment was chosen and described in order to bestexplain the principles of the invention and its practical application tothereby enable others skilled in the art to best utilize the inventionin various embodiments and various modifications as are suited to theparticular use contemplated. It is intended that the appended claims beconstrued to include other alternative embodiments of the inventionexcept insofar as limited by the prior art.

1. A method, in an image scanner, for detecting a defect, comprising:determining whether a line is present in image data for a first colorchannel; determining whether the line is not present in image data for asecond color channel, where the spectral bandwidths of light received byphotosensors for the first and second color channels are identical; anddetermining that the defect is on a calibration target when the line hasan intensity that is greater than a predetermined intensity threshold.2. The method of claim 1, further comprising: determining that thedefect was present during calibration when photosensors detecting theline have a Photo-Response Non-Uniformity intensity that is greater thana predetermined intensity threshold and a Photo-Response Non-Uniformitygain that is greater than a predetermined gain threshold.
 3. The methodof claim 1, further comprising: determining that the defect was notpresent during calibration when photosensors detecting the line have aPhoto-Response Non-Uniformity intensity that is greater than apredetermined intensity threshold and a Photo-Response Non-Uniformitygain that is not greater than a predetermined gain threshold.
 4. Ascanner, comprising: a first line-array of photosensors; a secondline-array of photosensors, where the first and second line-arrays ofphotosensors receive spectral bandwidths of light that are substantiallythe same; a processor; and the processor determining that a defectexists when a line is present in image data for only one of the firstand second line-arrays of photosensors; a calibration target; and theprocessor determining that the defect is on the calibration target whenthe line has an intensity that is greater than a predetermined intensitythreshold.
 5. The scanner of claim 4, further comprising: the processordetermining that the defect was present during calibration whenphotosensors detecting the line have a Photo-Response Non-Uniformityintensity that is greater than a predetermined intensity threshold and aPhoto-Response Non-Uniformity gain that is greater than a predeterminedgain threshold.
 6. The scanner of claim 4, further comprising: theprocessor determining that the defect was not present during calibrationwhen photosensors detecting the line have a Photo-ResponseNon-Uniformity intensity that is greater than a predetermined intensitythreshold and a a Photo-Response Non-Uniformity gain that is not greaterthan a predetermined gain threshold.